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How close are Galerkin eigenvectors to the best approxi- 
mation available out of the trial subspace ? Under a vari- 
ety of conditions the Galerkin method gives an approximate 
eigenvector that approaches asymptotically the projection of 
the exact eigenvector onto the trial subspace - and this oc- 
curs more rapidly than the underlying rate of convergence of 
the approximate eigenvectors. Both orthogonal-Galerkin and 
Petrov-Galerkin methods are considered here with a special 
emphasis on nonself adjoint problems. Consequences for the 
numerical treatment of elliptic PDEs discretized either with 
finite element methods or with spectral methods are discussed 
and an application to Krylov subspace methods for large scale 
matrix eigenvalue problems is presented. New lower bounds 
to the sep of a pair of operators are developed as well. 
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1 Introduction 

Consider the eigenvalue problem for a linear operator A 

Find A G C and v ^ so that 
Av = Xv 

We seek a family of approximations {Xh, Vh}h>o using the Galerkin method. 
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burg, VA 24061 USA. 



(1.1) 



The Galerkin method approximates the operator A with a finite rank (or 
in finite dimensions, a much lower rank) operator, - the "projection" of 
A, that samples the action of A on a subspace. The solution to (1.1) is then 
approximated with a matrix eigenvalue problem associated with A^. 

This work focuses on one particular bit of Galerkin folklore - "the Galerkin 
method yields an approximate eigenvector for A that is essentially the projec- 
tion of the exact eigenvector v onto the trial subspace." We discover that this 
statement is correct under some mild conditions if 1) "essentially" is taken 
to mean "asymptotically," and 2) the projection involved is intrinsic to the 
Galerkin method and may be either orthogonal or oblique depending on how 
the discretization is organized and what point of view is taken. Although more 
generality is possible, we restrict ourselves to a Hilbert space setting - specific 
assumptions are found in Section 2. 




Figure 1. How close is the approximate eigenvector Vh 
to the projected exact eigenvector P^v ? 



The basic features of Galerkin methods that play a role in our analysis 
are reviewed in Section 3. Of particular note here is that discussion is not re- 
stricted to self-adjoint problems. Section 4 provides analysis for the simplest 
case - when j4 is a bounded operator. The case where A is an unbounded 
operator is considered from two vantage points in the next two sections: with 
respect to the "energy" norm in Section 5 where a discussion of consequences 
for the finite element method on elliptic problems may be found; and with re- 
spect to the underlying Hilbert space norm in Section 6 where an elliptic prob- 
lem dicretized using a spectral method is discussed. The final section presents 
an application to the biorthogonal Lanczos method and Arnoldi method. 
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2 Setting of the Problem 



2.1 Operators defined via quadratic forms 

Although eigenvalue problems are most naturally posed for linear operators, 
the operators themselves are often difficult to specify fully - particularly with 
regard to the operator's precise domain of definition. It is often easier to char- 
acterize an operator in terms of a quadratic form that is naturally associated 
with it. This approach usually leads spontaneously to the appropriate choice 
of underlying Hilbert spaces. The reader may refer to the excellent tract of 



Kato 1 11 1 for background material on quadratic forms. 

Let Tihe a complex separable Hilbert space with inner product]^ and norm 
denoted by ( • , • )^ and || • ||^, respectively. Let a(-, •) be a closed sectorial 
sesquilinear form, densely defined in Ti.. "Sectorial" means that 

a{v,v) > aWvW^ 
|9m a{v,v)\ < M(3f?e a{v,v) - a||t;||^) ^ ' ' 

for all V £ Dom{a) and some fixed a, M > 0. Following the notation of Kato 



[11 1, define symmetric sesquilinear forms associated with a, 

(2.2) 



[3f?e a]{w,v) = ^(a(ii;,f) +a{v^w)) 



a\{w,v) = ^{a{w,v) — a{v,w)), 
so that a{w,v) = [5Re a\{w,v) + i [Qm a\{w,v). Notice that (2J.) implies 



a\\v\\l^ < I'Ste a]{v,v) < \a{v,v)\ < vT+iW^ [3?e a]{v,v), 

so [5Re a] is a closed, symmetric, positive-definite, sesquilinear form that in- 
duces an inner product on Dom{a) with respect to which Dom{a) is a Hilbert 
space. a(ti, v) is then a bounded sesquilinear form on this Hilbert space. 
Furthermore, there is a closed operator, Cq, densely defined on TL so that 
Dom{Ca) = Dom{a) and [^e a]{v,v) = WCavW^ (cf. |li, p. 331). 

Suppose now that V = Dom{a) is equipped with an inner product { ■ , ■ )^ 
equivalent to [5Re a]. The Hilbert space V is continuously and densely imbedded 
in Ti and we may assume without loss of generality that < for all 

u£V. 

Observe that any T^-bounded linear functional on 7i may be viewed imme- 
diately as the extension of some V-bounded linear functional on V, so letting 
V' denote the dual space of V, the imbedding V ^ Ji may be extended to a 
Gelfand triple (see e.g., [|l8[) V ^ H ^ V' with the norm on V' defined by 

II II \{w,v)'h\ 

||U||V' = sup T T . 

w£V \\w\\v 



Inner products and sesquilinear forms are conjugate linear in the first argument. 
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The Cauchy-Schwartz inequality yields ||v||v' ^ ll^ll?^ for all v 7i. 

Under the hypotheses given, Kato's first representation theorem ( [pl| , p. 
322) guarantees the existence of a closed m-sectorial operator, A, defined on 

Dom{A) = {v £ V \ \a{w, v)\ < my\\w\\fi for all w G Dom{a) } 

where is independent of w but will generally depend on v). Then 

a(n, v) = (u, Av)y^ 

for all V G Dom{A) and u G V. Furthermore, there is a closed operator Ba 
with Dom{Ba) = Dom{Ca) = Dom{a), so that A may be decomposed as 
A = B*Ca (see e.g., fT^l p. 337). "*" denotes the W-adjoint. 

Since Dom(A) is dense in V (with respect to the V-norm) and 

\a{u, v)\ < cllully ||f lly, 

we may calculate for any v G Dom{A) 

II ^ II \{u,Av)'h\ 

\\AV\\\!' = sup 7 7 < C||f||y 

m6v II^IIv 

Thus A may be extended by continuity to a bounded linear transformation 
from V to V'. Furthermore, 



I . 11 \o-(u,v)\ \a(v,v)\ a\(v,v) , 

\Av\\v' = sup ' , , ^' > ' , , ^' > — „ , ^ > allwl 
«ev ll^^llv FIIv II^IIv 



v 



thus A is an isomorphism of V onto V' with a bounded inverse, T = A~^ 
mapping V' back to V. T may be defined alternatively for each v G V, so that 
G V solves 

(u, u)^ = a{u,Tv) (2.3) 

for all u £ V. A maps vectors in Dom{A) to 7i, whereas T maps vectors in 
ncV back to Dom{A) C V. 

2.2 The gap 

Given two closed subspaces, Ai and M of the proximity of one to the other 
is measured in terms of the containment gap (or just ^apQ), which we define 
as 

6^{M,M) = sup m^^^^^^ = sm{emaxiM,^^)). 



^Kato defines the gap as max[5(X,7V), 5{M,M)\ 
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Qmaxi-M.,-^) is the largest canonical angle between M. and a "closest" sub- 
space of AA having dimension equal to diniA^. Notice that if diniAA < 
dimM then 5„(A^,AA) = 1 and 6^{M,M) = if and only if M C Af. If 
diniAA = dimM then 5^ (A4 , AA) = 5^{M,M). Conversely, if both 5^ ( A// , AA) < 
1 and 6y^{M,M) < 1 then 5^{M,N) = 6^{M,M) and dimAA = dimM. 

The gap can be expressed directly as the norm of a composition of pro- 
jections, so that if Hm and Ilj\f denote W-orthogonal projections onto M. and 
AA, respectively, then 6y^{M,M) = ||(/ - n_v)n;K||^. 

If A4 and M are closed subspaces of V, we have the completely analogous 
definition of gap relative to V: 

6v{M,Af) = sup inf 



cgX yeAf \\x\ 



V 



If A and B are closed operators in TC the gap between A and B is defined 
as the gap between their graphs, considered as subspaces oiTL xTL: 

d^{A,B)= sup mf — — ^. 

xGDom{A) y&Dom{B) \\x\\^ + 

2.3 The eigenvalue problem 

Our focus rests on the (weakly posed) eigenvalue problem for a: 

Find A and 7^ ?) G Dom{a) so that 
a{w.,v) = X{w,v)^ for all w G Dom{a). 

Note that {A, is an eigenpair for (2.4) if and only if -u G Dom{A) and {A, v} 



(2.4) 



is an eigenpair for the operator A; or equivalently when A 7^ 0, if {A~^, v} is 
an eigenpair for the operator T. 
Denote the resolvent set of A by 

p{A) = {z £ C\z — A has a bounded inverse on 

and the spectrum of A by (7{A) = C\p{A). A is an isolated eigenvalue of ( |2.4[) if 
there is a neighborhood of A, call it ^2(A), so that Q{X) r\a{A) = {A} (i.e., A is 



an isolated eigenvalue of A). If A is an isolated nonzero eigenvalue of (2^) then 
Ker[A — XI] is the associated eigenspace. 

^ = U^i Ker[{A - XI)''] similarly 
will be the invariant subspace for ( |2.4D associated with A. No compactness 
assumptions have been made for either A or T, so a priori it may happen 
that ( |2.4D has no eigenvalues at all or those that it has may be embedded in 
essential spectrum (defined with respect to A) and not isolated. A has finite 
multiplicity m if dim U = m < 00. If A has finite multiplicity then there is 
a finite integer, r < m for which Ker[{A — Xiy] = Ker[{A — XIY~^^]. The 
smallest such integer is called the ascent of A. 
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Furthermore, if A is an isolated eigenvalue with finite multiplicity, then 
for each A; = 1, 2, . . . , {A — X)^ is a Fredholm operator with zero index 
implying nullity{A - X)^ = nullity{A* - X)^ for each k (cf. |]ll|, p. 239). In 
particular, A is an eigenvalue of A* with the same multiplicity and ascent as 
A. A "left eigenvector" associated with such a A E cr{A) may be characterized 
variationally as 

Find / n* G Dom{a) so that ,^ . 

a{ix*,v) = X{u*,v)y^ for all v G Dom{a). 



Note that u* is a solution for if and only if u* G Dom{A*) and {A, -u*} 

is an eigenpair for the operator A* . 

Henceforth we will assume that there is an isolated eigenvalue A 7^ for 
( |2.4D having finite multiplicity m with an associated maximal invariant sub- 
space U for which we seek approximations. We denote with U* the maximal 
invariant subspace for A* associated with A. 

The spectral projection for A onto lA is defined by the Dunford integral 



E= — f iz-A)~^dz. 



where F is a circle in C centered at A leaving the origin and all points of 
<j{A) other than A in its exterior. The complementary ^-invariant subspace 
is denoted = Ran{I — E). 

Notice that /x = 1/A will be an isolated eigenvalue of T also with mul- 
tiplicity m and the same m-dimensional invariant subspace U as for A. The 
spectral projection may be defined with respect to T as 

E = ±- [ {z-T)-'dz. 



E 



where S is a circle in C centered at /x leaving the origin and all points of cr(T) 
other than /i in its exterior. 



3 The Galerkin Method 
3.1 Discretization 

In order to approximate the eigenvalue A and its associated invariant subspace 
we introduce two finite dimensional subspaces Si^h C V and S2^h C V - the 

trial and test subspaces, respectively. Assume that d\m.Si h = dim52,/i *== 
N{h). Typically, the dimension N[h) is monotone increasing as the "mesh 
size" parameter h decreases. 
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The Galerkin method proceeds by solving an eigenvalue problem as in (2.4) 
for the form a restricted to the finite dimensional space S2^h ^ Si^h: 



Find Xh and ^ Vh £ Si^h so that 

a{u,Vh) = \h{u,Vh)^ (3.1) 
for all u G 52,^,. 

The name is sometimes further qualified as either an orthogonal-Galerkin 
method or a Petrov- Galerkin method depending on whether Si^h = S2^h or 
not. When A is self-adjoint and Si^h = 'S2,hi this approach is called the 
Rayleigh-Ritz method. 

For any given h, the computational realization proceeds by fixing bases for 
Si^h as 02, • • • , 4>N{h), and for S2,h as V'l, V'2, • • • , i^N(h)- The problem 
( p. ID is then reduced to resolving the generalized matrix eigenvalue problem 

A^y =XhBhy (3.2) 
where Ah = [a(^„ </.j-)] ^ C^^^)^^^'^) 
andB, = KV'., 0,)^] G C^^'^"^^'^. 

If an eigenvector y of (3.2) has components y* = {yi, y2, ■ ■ ■ , yN{h)}^ then 

the corresponding Vh that solves (|3.lD is represented as Vh = Ylf=i^ Vj'Pj- 

For any r G C, define = T + t, which may be defined variationally by 
analogy to (^) as that operator that satisfies 

{u, v)^ + ra(u, v) = a{u, Trv) (3-3) 

for all u, V £ V. Notice that {A, u} is an eigenpair for ( p.4| ) if and only 
if {(A~^ ~^ t)j is an eigenpair for the operator Tr and more generally, 
(7{Tt-) = o"(r) + r. T and T-r have the same invariant subspaces U associated 
with each of the eigenvalues and A~^ + r, respectively. The effect of 
a translation of cr(T) by r produces (from ( |3.3| )) a discrete problem with 
translated spectrum. Instead of (3.2), we have 

Ahy = Xh{Bh + TAh)y. (3.4) 

The approximate spectra produced by (3.2) and ( |3.4|) are related as A^^ = 
A^^ + T but eigenvectors and invariant subspaces are identical. Since our prin- 
cipal interest is in eigenvector approximations, choices for r are immaterial, 
and particular choices will entail no loss of generality. 

Assume that the following "discrete inf-sup" conditions are satisfied: 

inf sup \a{u,v)\'^= (3{h) >0. (3.5) 

MV = '^ \\v\\y = l 
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and 

inf sup \{u,v)n\'^= hh) >0- (3.6) 

Since diuiSi^h = dim52,h = N{h), these are equivalent to the complementary 
conditions, 

inf sup \a{u,v)\ = P{h) > (3.7) 

and 

inf sup \{u,v)\=P{h)>0, (3.8) 

ll''llw=lll"llw=l 

respectively. Condition (3.5) is the usual discrete inf-sup condition (cf. [^) 
and guarantees that is invertible for each h. Analogously condition (3.6) 
guarantees that B/j is invertible for each h. Either (3.5) or (3.6) will guarantee 
that the discrete eigenvalue problem (3.2) is well-posed and associated with a 
regular matrix pencil for each h > 0. 



3.2 Projections 

Define Ph'.V ^ Si^h as PhV = Yli^j=i a{'tpj,v)-fij(pi where [7^^] = Direct 
calculation verifies = Ph, hence Ph is a projection, albeit nonorthogonal 
typically. Ph maps each v G V to a unique vector, w'^ = PhV in Si^h that solves 

Find € Si^h so that , , 

a{u, V — w^) = for all u G S2,h- 

Ph arises spontaneously in discussing solutions to boundary value problems 
associated with a. For any given f £ Ti, the weakly posed boundary value 
problem 

Find V £ V so that 
a(zi;,-0) = {w,f)y^ for all w £V. 

admits a solution v which may be approximated with a Galerkin method 

Find Vh G Si^h so that 
a{w,Vh) = {w,f)^ for ah w £ S2,h- 

Exact and approximate solutions are related via Ph as Vh = PhV- 
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Along the same lines as above, define as Pj^u = Y^^j=i'^{''^^4'i)lij'^i- 
is a projection onto S2,h defined on V and = P^u solves, for any u G V, 

Find G 52,/i so that ('\^vC\ 
a{u — w^,v)=Q for all G 

Notice that (|3.9D and (|3.10| ) together imply for all u, v G V, 

a{u,Phv) = a{P^u,Phv) = a{P^u,v) 

That is, P^ is the "a-adjoint" of Ph- 
Now, for all u G S2^h and all v G 

{u, v)^ ={P^U,PhV)^ 

= aiPf:u,TPhv) 
= a{u,PhTPhv), 

so we have that Xh ^ and together solve ( |3.1| ) if and only if A^^ and Vh 

def 

constitute an eigenpair for Th = PhTPh- 

From (3.7), we find for any G V with ||P/jt)||y 7^ 0, 

n / / \a{u,Phv)\ \a{u,v)\ 
< p[h) < sup — — — = sup -— — — 

ll«llv=i ll"ilv=l 

^ ciIItjII^II-uII^ 
— — II „ II — - < Cl- 



u&S2,h \\Ph'v\\y \\Ph'v\\y 

ll"llv=i 

Thus, 

\\Ph\\v<ci/^ih). (3.11) 

Similarly from (3.6), H-P^Hy < ci//5(/i). The following result leads us to con- 
clude that both ||/ - PhWy < ci/f3{h) and ||/ - P^l].;; < ci/(3{h) as well. 

Lemma 3.1 If Z is a bounded (nonorthogonal) projection on a Hilhert space 
W, then \\I — Z\\y^! = ||.^||w- Furthermore, if H denotes the W-orthogonal 
projection onto Ran{Z), 

^ ||(/-Z)n||n; < ||(/-n)n||n; < ||(/-^)n||w (3.12) 



\Z 



Proof: The first assertion was proved by Kato (|]10|, p. 28). Since {I — Z) = 
{I-Z){I-U), \\{I-Z)u\\w < ||(/-Z)||w||(/-n)u||>v which then gives the first 



9 



inequality of ( 3.12| ). The second inequality of ( p. 12 ) is the best approximation 



property of orthogonal projections. ■ 

Hi^h and 112, h will always denote orthogonal projections onto Si^h and 
S2,h respectively. However, depending on the context, they will be considered 
either orthogonal in TC or orthogonal in V with no distinction in notation. 

Define Qh : H ^ Si^h as QhV = J2i'j=i{i'j,v)nlij(l>i where [7^^] = B^^ 
Qh has a natural extension to v £ V' so the composition of operators AQh : 
V — > V and QhAQh '■ V — > Si^h are each well defined. Since Qf^ = Qh, Qh 
is also a projection, but is "H-orthogonal if and only if Si^h = <S2,h- Qh maps 
each V £ H to a unique vector, tu" = QhV in Si h that solves 



Find w'^ S Si^h so that 
{u, V — w^)^ = for all u G S2^h- 



(3.13) 



Evidently, the W-adjoint Ql^ :H ^ S2,h has the form Ql^u = Yld,j=i{'i^ii'^)'H 
Q\ is a projection onto 52,^ and solves, for any u £ Ti, 



Find w'^ S S2^h so that 
{u — w^,v)^ = for all v G Si^h- 



(3.14) 



Now, for all u £ S2h and all v £ Si h, 



a{u,v) =a{Q*hU,Qhv) 

= {QhU,AQhv)^ 
= {u,QhAQhv)y^, 

so we have that A/^ / and Vh together solve ( |3.1[ ) if and only if \h and Vh 
constitute an eigenpair for Ah *== QhAQh- 

3.3 Convergence 

Convergence criteria may be framed either in V or in "H. Convergence criteria 
in V appear as 

lim /3(/i)~^ inf ||f — w|L = for each u G V, (3.15) 

and 

lim5^(r,„r) = 0, (3.16) 



Theorem 3.2 ( Descloux, et al. [|, |6|]) The hypotheses (3.15) and (3.16) 
imply that 
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1. Both Ph I and —> I strongly in V; Ph is uniformly V-bounded with 
respect to h; and there is a constant c > so that 

6y{Th,T) < \\{I-Ph)TPh\\^<c6y{n,T). 

2. For each compact subset, IZ, of p{T) there exists Hq > and K > so 
that TZ C p{Th) and \\{z — Th)~^\\y < K uniformly for z £ TZ for all 
h < ho- 

3. If p is an eigenvalue ofT with algebraic multiplicity m and with an asso- 
ciated m- dimensional invariant subspace U, there will he m eigenvalues 
(counting multiplicity) of T^, {p\, /U™} that are convergent to 
II as h ^ and the associated m-dimensional T^-invariant subspace Uh 
satisfies 5{UMh) as h ^ 0. 

If S is a circle in C centered at p leaving the origin and all points of cr{T) 
other than /x in its exterior, then under the convergence assumptions (3.15) 
and (3.16), there will be m eigenvalues of T^, labeled as . . . , p^i 

that will all be contained in the interior of S for sufficiently small h. Thus for 
sufficiently small /i, the Dunford integral 



defines a spectral projection onto the T/j-invariant subspace Uh associated with 
Analogous convergence criteria in 7i appear as 

lim B(h) inf \\v — w\\n, = for each v £ H, (3.17) 

and 

lim<5^(A,A) = 0, (3.18) 
with similar consequences: 

Theorem 3.3 ( Descloux, et al. [|, |§) The hypotheses (3.17) and (3.18) 
imply that 

1. Both Qh I and I strongly in TL; Qh is uniformly TC-bounded 

with respect to h; and there is a constant c > so that 

6T^{Ah,A) < -Qh)AQh\\n < c6^{Ah,A), 

(if A is unbounded the second inequality holds trivially). 
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2. For each compact subset TZ of p{A), there exists /iq > and K > so 
that TZ C and ||(z — yl/i)~^||^ < K uniformly for z £ TZ for all 
h < Hq. 

3. If X is an eigenvalue of A with algebraic multiplicity m and with an asso- 
ciated m-dimensional invariant subspace U, there will be m eigenvalues 
(counting multiplicity) of A^, {Aj^, A|, A^} that are convergent to 
X as h ^ and the associated m-dimensional A^-invariant subspace Uh 
satisfies d-HiUMh) as h ^ 0. 

If r is a circle in C centered at A leaving the origin and all points of cr{A) 
other than A in its exterior, then under the convergence assumptions (3.17) 
and (3.18), there will be m eigenvalues of A^, labeled as A^, A|, A^, 
that will all be contained in the interior of T for sufficiently small h. Thus for 
sufficiently small h, the Dunford integral 



Eh = — I (z- AhY^dz 



defines a spectral projection for Ah onto the invariant subspace lAh associated 
withAi, XI ... X^. 

It will be convenient to label the complementary nonzero part of the spec- 
trum of Th and Ah as a''{Th) = a{Th)\{Q, nl, /i^Tl and a^iAh) = 
a{Ah)\{0, Xl Xl A;[^}, respectively. 

Theorem 3.4 (Babuska and Osborn Suppose the convergence hypothe- 
ses (3.17) and (3.18) hold. Then 



(3{hr^6^{U,Si,h 



l/a 



(3.19) 



|A - Xh\ < c 



f}{hr^K{l^\S2,h)K{U,Si 



l/a 



(3.20) 



These appear as Theorems 8.3 and 8.4 in |2|. As stated there, the proofs 
given in ||2| presume that A is compact - however, the arguments extend with- 
out change to unbounded A having nontrivial essential spectrum once the 
convergence results of H come into play. 



4 Bounded A 
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4.1 Basic Estimates 



Define 

^■u{h) = sup 



\\QhA{I - Qh)u\\^ 



where Hi h here is the W-orthogonal projection onto Si 



h- 



Theorem 4.1 Suppose the convergence hypotheses (3.17) and (3.18) hold. 
There exists an Hq > sufficiently small so that for each h < ho and each 
u gU, there is a Uh G Uh so that 

\\uh - Qhu\\-H < ce^(/i) 6y^{U, Si^h) (4.1) 

where c > is a constant independent of h and independent of the choice of 
uGU. 

Proof: Ran{Eh) C Si^h since E^Ah = A^E^. Note also tliat Qh is a spectral 
projection for Ah associated with all nonzero eigenvalues ol Ah. Thus, Qh — E^ 
is a spectral projection for Ah onto lif^ associated with all nonzero eigenvalues 
of Ah distinct from A. Let A = A\y denote the restriction of A to ^ and 
Let = Ah\u^ denote the restriction of A^ to Uf^. Then, A'f^{Qh — Eh) = 
{Qh — Eh)Ah and we have 

AliQh - Eh) L - {Qh - Eh) L A ={Qh - Eh){Ah - A) L 



-{Qh - Eh){{I - Qh)A + QhA{I - Qh)) 



u 



-{Qh - Eh)QhA{I - Qh)\ 
-{I -Eh)QhA{I -Qh)\u 



Thus, the mapping S : U ^ given hy S = {Qh — Eh) \y is a solution to the 
Sylvester equation, 

AlS-SA = -{I-Eh)QhA{I-Qh)\u (4.2) 
There exists Ki > Q such that 

^-l I, . II / 11^-, A\-l\ 



A)-'\u ||„<||(z-^)-^||^<i^: 



uniformly for all z G F. Likewise there exists an /iq > and K2 > such that 
for h < ho, 



A'h)-' \wJ\n<\\{z-Ah)~'\\n<K2 



uniformly for z G F. Therefore, the pseudospectral sets A^{Ah) are contained 
in the exterior of F for any e < 1/K and for all h > ho. By Lemma A. 1K b), 
there must then be a cq > independent of h, such that 

\\{Qh - Eh) \u Wu^w^ < CO \\{I - Eh)QhA{I - Qh) \u Wu-^ui- 



13 



Thus, for any u gU, 

urn TP \ n ^ \\t tp n \\QhA{I - Qh)w\\v 
UQh - Eh)u\\j^ <co ||/ - EhW^ sup — 

<C0 \\Eh\\u sup — y— — sup 

■wdU \\V-^\,h)w\\y^ y,^u \\w\\h 

=co \\Eh\\-H^H^h) 5^{U,Si^h)- 

Notice that since converges uniformly to WEhW-y^ is uniformly bounded. 
The conclusion follows upon assigning = E^u. ■ 

Corollary 4.2 Suppose the convergence hypotheses (3.17) and (3.18) hold 
and that Si^h = 52, ft- 

S^{U,Si,h) < ^ni^Mh) < {l + ce^{h))6^{U,Si,h) (4.3) 

Proof: Note that under the hypotheses given, Qh = I^i^h = 1^2, ^ = Q^- 
The first inequality of (|4.3|) follows trivially from observing that Uh C Si^h- 



For the second, by Theorem 4.1 there exists an Hq > such that for each 
h < Hq and u £h( with = 1, there is a G such that \\Q hU — UhW-j^^ < 
ce^{h)5^{U,Si^h)- Then, 

min ||n — n/ill^ <||u — -u/ill^ 

<\\u - Qhu\\y^ + WQhU - UhWu 

Maximizing over u yields the conclusion. ■ 



Corollary O can be interpretted as saying that, provided E^{h) — > 0, the 
orthogonal Galerkin method provides optimal zero order approximations to the 
eigenvectors of A: Uh will approach asymptotically the closest m-dimensional 
subspace in Si h to the exact eigenspace U ~ and this is true even if A is 



nonselfadjoint. See the comments following Theorem 5.4 



4.2 Related Estimates and Interpretation 

Since each Qh is a bounded projection, Tl may be decomposed into a direct sum 
of complementary subspaces as H = Ran{Qh) © Ker{Qh) = Si^h © •^i'h- '^^^ 
operator A can then be partitioned in a way that reflects this decomposition 
- see Figure 2. 

For each generalized eigenvector, u € U, define u^^'^^ = QhU and n*-^'''^ = 
{I—Qh)u. The quantity then is a measure of the relative size of QhA{I— 

Qh) on u^'^''^\ This follows from observing that from Lemma |3.1| 



\\QhA{I - Qh)u\\) ^ _ ^ „^ „_ \\QhA{I-Qh)u\\) 



fTTT 7m - ^n(^) ^ WQhW sup — — — 

[i-QhM uGU \\[l-Qh)u\ 
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which (provided Qh — > / strongly) imphes immediately 



\\QhA{I-QH)n^^^^Hn ^ \\QhA{I - Qh)u^^^^H-^ 



«{2,fe)gA6,{W) 11^ ' \\h u(^''^)eNh{U) IF' ' '\\h 

for some constant c > 0, where Mhipi) = {I — Qh)U C Mh{U) is the 

span of all components u^"^'^^ of (generalized) eigenvectors u mU that lie the 
direction of Ker{Qh). 



r 




-r -\ 
1 1 




1 — — 1 

1 1 




Ah 


' QhA{I-Qh) ' 




1 1 


h 




1 1 

+ H 

1 1 


u = 


1 1 
1 — 1 
1 1 




{I - Qh)AQh 


' (/ - Qh)A{I - Qh) ' 




'„{2,h)' 


L 




1 1 

_L J 




1 1 
1 1 



Figure 2. Partitioning A and u gU onTL = Sih 
Shading indicates influence on e^{h) 



Certainly it may happen that E^{h) 7^ 0, so a variety of additional con- 
ditions will be examined in the next few sections that suffice to guarantee 
e^(/i) 0. Perhaps the simplest of these is to require that A*j^ eventually 
converge to A* in gap: 

Theorem 4.3 There is a c> such that e^{h) < c5^{A*^, A*) 
Proof: Note that I -Qy, = {I - Qh){I - lii^h)- Thus, 

WQhAjl - Qh)u\\^ 



sup 



I - ^i,h)u\\j^ 
<\\QhA{I-QH)\\n = W-Ql)A*Ql\\-n 
<c6-^{Al,A*), 



using Part 1 of Theorem |3.3| 
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We shouldn't expect to do much better than the bound provided by (4.1). 
The bound has the "right" asymptotic behaviour in many cases and so in that 
sense will be best possible. 

Theorem 4.4 Suppose that X is a simple eigenvalue of A with an associated 
eigenvector u. There exist constants cq, ci > (independent of h) so that for 
each h one may find a Galerkin eigenvector Uh with 

coWuh - Qhu\\j^ < \\QhA.[I - Qh)u\\-j^ < ci\\uh - Qhu\\-^ + |A - \h\ \\uh\\-y^ 

(4.4) 

Furthermore there exists a C2 > independent of h so that 

^JTt TT ^ II - W < ClJT- „ . II + C2/0 I - Ii2,h)u ^ 

-III ^h)u\\y^ ^ -jii^h)u\\-j^ m ^ 



where u* is a "left eigenvector" satisfying l[2.^) . 

Proof: The first inequalities in each of (4.4) and ( [4. 5]) are a consequence of 
Theorem [4.1| . Since A is simple, Au = Xu and for h > sufficiently small, 
rank{Eh) = 1 and Xh will be simple. 

\\QhA{I - QhMt^ <\\{I- Eh)QhA{I - Qh)u\\^ + \\EhQhA{I - Qh)n||„ 
<||(i^^5 - SA)u\\^ + \\EhAu - EhAhuW^ 

< (Phllw + 1^1) II^^IIh + W^^hu - AhEhuW, 



<{\\Ah\\H + \M)\\Sn\\^ + \\{X-XH)E^,n\\^ 



<\\Mn['^ + \\Qh\\n) \\EhU-Qhu\\^ + \X- Xh\\\Ehu\\T^, 



where S = {Qh — Eh) \^ satisfies the Sylvester equation ([4.21) . ( [4.4]) then 
follows upon assigning Uh = E^u and observing that Qh is uniformly bounded 
inn. 

To show ([4.5|), note that the ascent of A is 1, so from ( 3.2C| ) there is a C2 
with 

|A - Xh\ < C2m-^\\il - T^2,h)u*\\^\\{I - HhM^ 

so 

_ \\QhA{I - Qh)u\\^ 

ll(/-n.)n||^ 



\uh - Qhu\\n 
||(/-n;,)'u||^ ' '^"'m' 



<^ Uur + (c2/o_)||(/ - ^2,h)u*\\T^ 
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5 Unbounded A — Estimates in V 



5.1 Basic results 

The setting considered in this section is the traditional one encountered in the 
analysis of finite element methods. With few exceptions, much of the structure 
of arguments of Section 4 carry over into this setting. 
Define 

||FjT(/ - P,Ju||v 

^""•''S ll(/-n,,.)u||, 

where Hi^h now is the V-orthogonal projection onto Si^h- 

Theorem 5.1 Suppose the convergence hypotheses (3.15)-(3.16) hold. There 
exists an Hq > sufficiently small so that for each h < Hq and all u (zU, there 
is a Uh £ Uh so that 

\\uh - Phu\\v ^ cey{h)5y{U,Si^h) (5.1) 

where c > is a constant independent of h and independent of the choice of 
ueU. 

The proof is the same as for Theorem |4.1| formulated in the Hilbert space V 
instead of 7i and with T, Th, and Ph playing the roles of A, Ah, and Qh, 
respectively. 

Lemma 5.2 Suppose the convergence hypotheses (3.15)-(3.16) hold. If V is 
compactly imbedded inTi (so that T is compact as a mapping from V to itself) 
and if S2^h satisfies the approximation property 

lim (3{h)^^ inf \\v — w\\y = for each v G V, (5-2) 
then £^{h) as h ^ 0. 

Proof: Since a is bounded and coercive on V, there is a bounded and invert- 
ible linear operator on V, A, such that a{u,v) = {u,Av)^. Let "★" denote the 
V-adjoint and observe that P^u = Yldj=i{'i^ii^)\nij^*^v ^° that, in particu- 
lar, Ran{P^) = A*S2^h- Let Il2,h denote the V-orthogonal projection onto S2^h 
and notice that A*Il2,h-^~* is a projection (no longer orthogonal, in general) 
onto Ran{P^). Then for any ti G V, (|5.2| ) implies 

||(/ - P^hhh = \\il- - A*Il2,hA~*)u\\y 
= \\{I - P^h)A'{I -Ii2,h)A-^u\\^ 

< \\I - PtWyWA'^WyW - Ii2,H)A-*u\\^ 
C 

m 



<^||(/-n2,,)-A-*^x||^ 



—— inf \\A'*u-w\\y^Q, 
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for some constant c. Thus, converges strongly to 7 in V. Since T* is 
compact, 11(7 - P^)r*||^ and 

e (h) - sup \\P>^ni-PH){I-Il,,,)u\\y 

< ||P^T(7-P;,)||^ 

= \\{i-p^)T*pnv 

<\\{I-Ph)T*\\y\\Pnv^^ ■ 

Even when T is not compact, additional conditions on S2,h can yield the 
same result: 

Lemma 5.3 Let T* denote the H-adjoint ofT. Suppose the convergence hy- 
potheses (3.15)-(3.16) hold. If S2^h satisfies the approximation properties 

lim. f3{h)~^ inf — i(;|L = for each v^V, (5-3) 

and 

sup inf \\T*v -w\\y'^= -f{h) as h ^ 0. (5.4) 

\\v\\v=l 

Then e^{h) ^ as h ^ and ey{h) = 0{-i{h)). 

Proof: We first verify that T* maps V to V. For any u e H, v e Dom{a), 
\{u, v)y^\ < ^\\u\\^ \\Cav\\-f^. Thus, for u G Dom{a), w G Ran{Ca\Dom{A)), (so 
that w = CaV for some v G Dom{A)), one may observe 

\{T*u,B*w)^\ = \{T*u,B*Cav)^\ 
= \{T*u, Av)^\ 

Since Dom{A) is a core for Ca, Ro,n{Ca\Dom{A)) is dense in Ti. and as a con- 
sequence T*u G Dom{B**) = Dom{Ba) = Dom{a). For u G Dom{a), v G 
Dom{A), then: 

a(r"'u, v) = a{u, Tv) ={u, v)^^ (5.5) 
={u, TAv)y^ 

={T*u, Av)^ = a{T*u,v), 
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(the last equality being a consequence of T*u G Dom{a)) and so, T"" = T*. 

Since || • ||y and [3^ea] are equivalent norms on V, there is an m > so 
that m||u||^ < 10(^,^)1 and 

n^WPTd P^vW < HPhni-Ph)v, PhT{I-Ph)v)\ 

< sup \a{u, PhT{I-Ph)v)\ 

l|w|lv=i 

= sup \ai{I-P^)T*P^u, v)\ 
ll«llv=i 

< sup ci\\v\\J{I-P;i)T*Pliu\\^ 

uev 

= cMj{i-p(:)T*pnv 

Thus, e^{h) < \\PhT{I-Ph)\\y < - P,^)r*P,^||^. Now, notice 

that RaniPli) = S2,h, so / - = (/ - P^")(/ - U2,h) - where U2,h is the 
V-orthogonal projection onto S2,h- Now, 

11(1 - P{l)T*P;i\\^ < 11(1 - P,»)(/ - U2,H)T*U2,HPnv 

<\\iI-Pli)UP;i\\JiI-U2,H)T*U2,H\\y 

= m - Ph)\\v\\Ph\\v sup inf \\T*v-w\\^ 
\\v\\y=l 

< c-/{h) ^ ■ 

If a is symmetric (so that [?R.e a] = a and [9m a] = 0) and if a(-, •) itself 
is used as the inner product for V, then T is a self-adjoint operator in V. If 
additionally Si^h = S2,h^ then P^ is a V-orthogonal projection and is then 
also sclf-adjoint. In this circumstance, Uh is asymptotically the closest vector 
out of Si^h to u (with respect to the a-norm on V): 

Theorem 5.4 Suppose a is symmetric, a{u,v) = {u, v)y, and that Si^h = 
S2,h- Then 

Syih) < ciS^{Th,T) (5.6) 

and 

l<P§^<l + C2S^{n,T) (5.7) 
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Proof: Note that under the hypotheses given, Ph = lii^h = ^2,h = Ph- Thus, 

||ni /jr(/ - ni./j)n||v 
£^(h) = sup — — — - — — — 

u&A W -^^i,hM\v 

< \\Ui^hT{I-U^,h)\\v 

= ||(/-ni,;,)Tni,;,||^ 

< ci6y{Th,T) 



The first inequahty of (5.7) follows trivially from observing that Uh C Si,h- 



For the second, by Theorem p.l| there exists an /iq > such that for each 
h < ho and u G U with \\u\\y = 1, there is a Uh £ Kh such that \\PhU — Uh\\^ < 
ce^{h)dy{U,Si^h)- Then, 

min \\u - UhWy <\\u - Uh\\-^ 

Uh&^h 

<\\U - PhU\\y + \\PhU - -Uhllv 

Maximizing over u yields the conclusion. ■ 

Theorem was essentially given by Chatelin |Q] , refined by Babuska and 
Osborn Q - each for compact self-adjoint T. Recently, a more general result 
of this sort allowing for noncompact self-adjoint T was given by Knyazev 

5.2 Elliptic boundary value problems: Finite Elements 

Let be a bounded open subset of with a boundary dVL that is at least 
(jr+i,i £qj. gQjj^g integer r > 0. Given real coefficient functions ajj, hi, c (z 
C"'(jl), consider the second order elliptic differential operator A defined by 



Djaij{x.)DiV + ^ bi(x)DiV + c(x)f in O 

i, j=l 1=1 

with z; = on dQ, and the related adjoint operator given by 

n n 

A*{x,D)u = — Diaij{x)DjU — Di{bi{x.)u) + c{x.)u in ft 

i, j=l 1=1 

with n = on dfl, 

Suppose that A{x, D) is uniformly strongly elliptic. The associated bilinear 
form: 

n „ n „ 

a{w, v) = 2, / a,ij{'^)[Djw][Div]d:>c +y. / w bj{^x)[Djv]d:>^ 
,,j=iJ^ j=iJ^ 

+ / c(x) w V dx, 
Jn 
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defined on Dom{a) = Hq{^) is a closed sectorial bilinear form densely defined 
inTi = L'^(Q) and A is manifested as a densely defined m-sectorial operator on 
Tl which can be extended to a continuous bijection from V = Hq{^) onto the 
dual space V' = H~^{Q). Here and elsewhere, 77^(^7) denotes the completion 
of the vector space C°° with respect to the norm 

\W\\Hp{n) = yZ \D"v\^d:>c. 
r~z Jn 

\a\<p 

The associated seminorm is defined as 

\a\=p 

Results governing regularity of solutions to elliptic problems (e.g., p. 
328) guarantee for any / G H^'~^{i^), the weakly posed problem 

a{w,v) = {w,f)L2(^Q^ for all w e H^{^) (5.8) 

has a solution v G H^'~^^{il,) n Hq{Q). Since T is a closed mapping from 
H^~^{^) to H^'~^^{^1), there exists a constant c > such that ||r/||j|^r+i(f2) < 
c||/ll_ff''-i(r2) foi' s-ll / £ H^~^{^). Furthermore, ifU denotes an invariant sub- 
space of A associated with an isolated eigenvalue of A with finite multiplicity 
thenZ^ C H''+^{n). 

Likewise the adjoint problem, 

a{u,'w) = {w, g) i2(^Q-^ ioT all w £ (Q) (5-9) 

has a solution u £ H^^^{Q) for any g G H^^^{il). So, in particular, if 51 G V C 
H^{n), then T*g = iA*)-^g G H^{n). 

Apply the Galerkin method with Sh = Si^h = 'S2^h chosen to be a family of 
finite dimensional subspaces of V, so that for all integers < A; < r and some 
fixed 0, u £ H^^^{Q,) implies 

For example, C'^-finite element spaces constructed from piecewise polynomials 
of degree at least r would satisfy this condition. 

The discrete inf-sup condition (3.5) is satisfied with P{h) = a > 0. Thus, 
the convergence condition 3.15 is immediately satisfied. It remains to verify 
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that (3.16) holds. Note that for every x ^ Sh = Dom{Th), Tx £ H'^{Q) and 

.^^ Ik - y\\HHn) + \\ThX - TyWiji^^) ^ \\ThX - Tx\\Hi(n) 
y€m{n) WxWh^u) + \\Thx\\Hi(n) ~ \\x\\h^q) + \\Thx\\m{n) 

UI - Ph)Tx\\Hiin) 

\\x\\m{Q) + \\Thx\\H-^{Q) 

\x\\m{n) + \\Thx\\m{n) 
ch^\\x\\Hi{n) 



< 



< 



xWhHQ) + WThxWni 



Thus, lim/,^0 S^iTh, T) = 0. 

Although Theorem is apphcable, Theorem 5.3 will yield a concrete rate 
once we evaluate 

7(/i) = sup inf ||T*w — wily 

l|f|lv=i 



For any v £ Sh, T*v G H^{Vl) and so 



inf ||r*?; — w 



|y = ||(/-n,)T*t;||y 



< ch^Wvl 



Thus, 7(/i) < c/i^. 

Since C H''+\n), 



6\;{ly(,Sh) = sup inf 



\y - x\\y _ \\{I - Ilh)x\ 



Theorem p.l| asserts that there exists an /iq > sufficiently small so that 
for each h < ho and all n G there is a Uh £ Uh so that 

\\uh-Phu\\y<cK-+\ 

whereas ||iift, — ti||y and ||n — -Pft,ii||y will each be only of order K' typically. 
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6 Unbounded A — Estimates in 7i 



6.1 Basic results 

In the V-setting explored in Section 5, orthogonality of Ph and the related 
best approximation property in V could be developed only for self-adjoint A. 
In contrast, estimates in H such as were found in Section 4 have particular 
appeal since whenever Si^h = S2,h, Qh will be an orthogonal projection in H, 
notwithstanding asymmetry in a and nonselfadjointness of A. Unfortunately, 
those estimates obtained in Section 4 depend fundamentally on the bounded- 
ness of A. In particular, if A is unbounded then e^{h) might not be uniformly 
bounded in h, much less go to zero. 

We define an expression that plays a role analogous to that of £-^{h) in 
Section 4: 

where Hi^h here is once again the 7i-orthogonal projection onto Si^h- 

Theorem 6.1 Suppose the convergence hypotheses (3.17) and (3.18) hold. 
There exists an ho > sufficiently small so that for each h < ho and all 
u & U, there is a Uh & Uh so that 

\\uh - Qhu\\u - ^^ni^) ^ni^^^hh) (6-2) 

where c > is a constant independent of h and independent of the choice of 
ueU. 

Proof: Since E^Ah = AhE^, Ran(Eh) C Si^h Note also that Qh is a spectral 
projection for A^ associated with all nonzero eigenvalues of A^. Thus, Qh — E^ 
is a spectral projection for A^. onto U^. Let A = A\u denote the restriction 
of A to U and Let = Ah\u^ denote the restriction of A^ to U^. Then, 
Al{Qh - Eh) = {Qh - Eh) Ah and we have 



AUQh - Eh) - (Qh - Eh) 1^ A ={Qh - Eh){Ah - A) 

= -{Qh - Eh){{I - Qh)A + QhA{I - Qh)) 
= -{Qh-Eh)QhA{I-Qh)\u 
= -{I - Eh)QhA{I - Qh)\u 



u 

j_ r). A( T — n.\\\ 

u 
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Now premultiply by T/j and postmultiply by T to find 

{Qh - Eh) \^f- n{Qh - Eh) \u=- Th{I - Eh)QhA{I - Qh)T \^ 

= -{I- Eh)ThQhA{I - Qh)T 
= - (/ - Eh){ThQhAT - ThQhAQhT) 



u 



{I-Eh){ThQh-QhT) 
{I - Eh){PhT - QhT) 



u 



Thus, the mapping S ■.U ^Uf^ given by 5 = {Qh — Eh) \y is a solution to the 
Sylvester equation, 



nS-ST={I-Eh){Ph-Qh)T 



u 



The goal now is to show that the bounds developed in Appendix A are 
applicable. There exists a Ki > such that 

\\{z-A)-^\u \\-H<\\{z-A)-^\\^<K, 

uniformly for all z G F. Likewise there exists an /iq > and K2 > such that 
for h < ho, 

\\{z-Al)-^ \u^J\^<\\{z-Ah)-'\\^<K2 

uniformly for z gT. Therefore, the pseudospectral sets Af:{Ah) are contained 
in the exterior of F for any e < 1 /K2 and for all h > ho. By Lemma A. 1K b) , 
there must then be a c > independent of h, such that 

\\{Qh-Eh) \y \\u-.u^^ < c\\{I - Eh)QhA{I - Qh)\u Wu^ui- 
Thus, for any u GU, 



\{Qh - Eh)u\\y^ <c\\I - EhW^ sup 



WQhAjl - Qh)w\\^ 



\\QhA{I - Qh)w\\T^ -Ui,h)M\v 
<c\\Eh\\T^ sup^- — I sup — 

=c \\Eh\\y^ e^(/i) 6^{U,Si^h)- 

Since Eh converges uniformly to E, is uniformly bounded. The con- 

clusion follows upon assigning Uh = EhU. ■ 

When A is unbounded on Ti. (so that V ^ 7i), \\Ph\\'}i will not typically be 
uniformly bounded with respect to h. Estimating the rate at which £-^{h) 
as /i ^ becomes technically more demanding and additional hypotheses are 
honorably acquired. For the remainder of the section we assume that 

Si^h = S2,h'= Sh = span{<j)i, 412, cl)N{h)} 
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and that 

The trial vectors (pi are eigenvectors of an auxiliary operator Aq. 



Theorem 6.2 Suppose that A is decomposable as A = Aq + B so that Aq is 
positive definite and self-adjoint on Dom^Ao) = Dom{A) and B* is bounded 
relative to Aq with relative AQ-bound less than 1. Furthermore, assume that A 
is a nondefective eigenvalue of A. 

1. If B* is relatively compact with respect to Aq then e^{h) — > as h ^ 0. 



2. If 



. \\B*V-w\\j^ def o 

sup mt — — — = "f[h) 



then e^(/i) ^ as h ^ and e^{h) < 0{'y{h)) as h ^ 0. 

Proof: Since B* has relative bound with respect to Aq smaller than 1, 
there is a r G C so that ||-B*(Ao — = 1 — k < 1 for some k > 0. It will 

be useful to translate the spectrum of ^ by r and write A — t = [Aq — t) + B. 
Referring to the discussion around ( |3.3[ ), we absorb this shift in spectrum 
into both A and Aq^ and assume without loss of generality that ||i3*^Q ""^H^ = 
1 — K < 1. Before continuing, we verify that the assumptions of (|l|) and (||) 
are preserved for any appropriate choice of r. First considering (|l|), if r is in 
the resolvent set for Aq then is in the resolvent set for ^ and 

B*{Aq - t)-i = ^B*A-\t-^ - A-^^)-\ 

B*Aq^ is compact, (t~^ —A^^)^^ is bounded, so B*{Aq — t)~^ is also compact 
and the assumption of (|^) will be independent of any feasible shift r. Likewise 
for there is an m > so that for any v G Dom{AQ), \\{Aq — T)f ||^ > 
mllwll^. Thus for any v € Dom(Ao)) 

\\Aqv\\^ < \\{Aq - t)v\\^ + |t| • \\v\\^ < (1 + ^)ll(^o - t)v\\^- 
If the assumption of (^) holds, then 

^(.Sh-^^Sh \\{Aq - t)v\\^ m 
so the assumption of <@ is independent of the selected shift r. 
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Now we prove (|2|) first. Write A = Aq{I + B) and observe then that 

ThQhA{I - Qh)Tu =PhTQhA{I - Qh)Tu 

=PhTAoQhAo^A{I - Qh)Tu 
=PhTAoQhAo'B{I - Qh)Tu 



Note that 



N{h) 

iPhTAovWl^ =11 ^ a{(l)j,TAov)'yij(pi\\'^ 

i,j = i 
N{h) N{h) 

i=l j=l 

-IIHxlP 



where 



1 1 



and G is tlie matrix inverse to 

= [{cl>e,iI + A^'B)cPk)y] 
=I+[{B*A^'(l)e,(l)k)n]- 
Observe that for any y G C^^'') with ||y||cJV(h) = 1, 

\\[{B*A^^cl>e,^k)nMc^w < P*^o'llw < 1 " 
hence we obtain a bound for P/^TAq that is uniform in h: 

1 1 



Since A is nondefective, XTu = u for any u £U and we find 

WThQhAjl - Qh)Tu\\-f^ 



^T-lih) =sup 

< sup 

< sup 



«GW \\{I-Qh)u\\n 

\\QhA^'B{I-Qh)Tu\\y 

\\QhAQ^B{I-Qf,)-iI-Qh)Tu\ 



n 



ueu k|A| • ||(/-Q;,)Tn||^ 

<^\\Qy,A-^^B{I-Qh)\\y. 
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Now if the condition of (|^) above holds then 



\\Qj,Aq^B{I - Qh)\\-H <\\Ao^B{I - Qh)\\T^ 
= \\{I-Qh)B*A^'\\^. 

Since B*Aq^ is compact and Qh — > I strongly, ||(/ — Qh)B*AQ^\\j_^ — > as 
h ^ from which follows the conclusion of (|l]). 

Now observe that A^^Sh = Sh so that, if the assumption (^) holds, then 

\\QhA,'B{I - Qh)\\T^ =\\{I - Qh)B*Ao'Qh\\^ 

. \\B*A^^u-w\\^ 
= sup inf — — 

. \\B*v-w\\n, 
= sup mf — — — 

and as a consequence, 



6.2 Elliptic boundary value problems; Spectral Methods 

Let fl be the unit square in M? : [0, 1] x [0, 1] and let there be given a 
function b = [6i(x), 62 (x)] G C^{n) x C^{n) such that b = on dn. Consider 
the differential operator given by 

A{x, D)u = -Au + b(x) • Vu + c(x)n in 17 

with u = on dQ. The regularity results of Kadlec |p, for example, show that 
Dom{A) = H^DH^. 

The application of a Fourier Galerkin method involves trial functions of the 
form (/)k(x) = sin(/i;i7rxi) • sin(/c2vrx2). If |k| = ki + k2 denotes the length of the 
multi-index k, parameterize the family of subspaces as Sh = span|i^|;j^|{(/)k}, 
and assign Sh = Si^h = S2,h- Define Aq = —A with Dom[AQ) = H'^[Q) n 
Hq{Q) and Do'm{AQ'^^) = V = Hq[Q). Observe that Aq is positive definite 
and self-adjoint inH = L'^{Q) and 

Ao(f)]i = \^(t>]i, 

with = (kf + A;|)7r2. If B denotes the closure of b(x) • V + c(x) on C^(ri) 
then the W-adjoint of B may be calculated as B*u = —V • (b(x)ti) = — b(x) • 
Vu + (c(x) — V • b(x))w and Dom{B*) D Dom{AQ). Since B* is compact 
relative to Aq , B* has relative ^do-bound of and Theorem |6.2| Part |^ asserts 
that £i^{h) — > 0. Furthermore, Theorem |6.2| Part |2| provides a mechanism for 
estimating the rate at which e^{h) — > 0. 
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First, notice that if {A, u} is an eigenpair for A, then 

-Au = {X-c)u-h-Vue Ho{n) = Dom{AQ^^). 

Thus, u G Dom{AQ^^) and 

\\iI-Qh)uf= K-^k, n)p 
|k|h>i 

Ik|h>l 



|k|/i>l 




The first inequality is a consequence of kl + kl> i|k|2 > l/(2/i2); the second 
is Bessel's inequality with respect to the orthonormal system {0k}- 

A similar argument can be organized to estimate 7(/i). For any v G L^(ri), 
A-^v e H^{n)nH^{n) and B*A-^v G H^{n) C Dom{Al^^). aI^'^B*A-^ is a 

closed, everywhere defined operator on L^(fi), hence Aq^^B*Aq^ is a bounded 
operator on L^(fi). 

Now for any v & H with \\v\\-^ = 1, 

inf ||SMo^-«;||5^= E K^k, SMoMi' 

|k|/i>l 

= E (^k)-'l('^k, aI^'b*a^\)\' 

lk|h>l 

|k|/i>l 

for some M < oo. From this we obtain 

j{h) < sup inf \\^*'^o 'y -M\H < 
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Theorem |6.1| asserts that there exists an /iq > sufficiently small so that 
for each h < ho and all n G ^, there is a u/j G Uh so that 



\\uh - Qhu\\-H < c/i^, 
whereas — and \\u — QhuW-j^ will each be only of order in general. 



7 Application to Krylov Subspace Methods 

Our previous results and examples were formulated in an infinite-dimensional 
setting and had (ultimately) an asymptotic character as the dimensions of the 
approximating subspaces grew without bound. Although, strictly speaking, 
such results are not applicable in finite-dimensional settings (e.g., if A is an 
n X n matrix), asymptotic results of this sort can be useful in understanding 
the characteristics of methods for calculating eigenvalues of very large matrices 
that are expected to give accurate results with low dimension approximating 
subspaces. We proceed with an analysis of the biorthogonal Lanczos method 
and of the Arnoldi method with this understanding in mind. Throughout this 
section TC = C", equipped with the usual Euclidean norm. 

Let A be an n X n complex matrix and let J be a tridiagonal matrix that 
is similar to A - so that A = VJV~^ for some nx n invertible matrix V. For 
any index 1 < i < n, \et Jg denote the ^th principal submatrix of J: 



ai 72 

P2 "2 73 

Pa "3 



Pi ae 



and define Ji+i via a partitioning of J as 



J 



* J 



Let = [vi, V2, ... , v^] denote a matrix containing the first £ columns of V; 
and for W* = V"-*^, let = [wi, W2, . . . , w^] denote a matrix containing 
the first i columns of W. Denote the remaining columns of V and W as 
Vf+i = [v^+i, V£+2, • • • , v„] and W^+i = [w^+i, we+2, ■■■ , w„] respectively 
so that, V = [ V^+i ] W = [ W^+i ] 



The Lanczos algorithm [13| builds up the matrices J, V, and W one column 
at a time starting with the vectors vi and wi. Only information on the action 
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of A and A* on selected vectors in C" is used. Different choices for vi and wi 
produce distinct outcomes for J, if all goes well. Recovering from situations 
where not all goes well is a fundamental aspect of later refinements of the 
algorithm (though we will not dwell on it) and two approaches to this are 
discussed in P] and |14]. 



At the Ith. step, the basic recursion appears as 

AV, =V,J, + (3i+ive+iel (7.1) 

Typically, normalization is determined so that |7j| = and v*Wj = 1. 
With exact arithmetic, the first £ — 1 steps yield matrices V^, that satisfy 
V|W, = I, 

Raniyn) =span{vi, Avi, . . . , A^^-'^vi} = /Cf(A, vi) 
Ran{We) =span{wi, A*wi, . . . , {A*Y~^wi} = /C^(A*, wi) 

Using the natural parameterization provided by i (instead of h), note 
that Qg = V^W| is a projection onto the subspace = span^^^ ei^k} 
and so Ai = Q^AQi = V^J^W^, and the eigenvalues for J^, the so-called 
"Lanczos eigenvalues", are Galerkin approximations to the eigenvalues of A. 
In particular, if J^z^ = A^z^^) and = V^z^ then {A^, u^} wiU be 
a Galerkin eigenvalue/eigenvector pair ostensibly approximating some eigen- 
value/eigenvector pair {A, u} of A. 

Theorem 7.1 Let X be a simple eigenvalue of A with an associated unit 
eigenvector u. Suppose the Lanczos eigenvalue A^^^ converges to X as i in- 
creases. There exist constants cq, ci > so that for each i > 1 the eigen- 
value/eigenvector pair {A*-^^, u*-^^} (with u^^^ appropriately normalized) satis- 
fies 

co||u(^) - Q^ull < |7^+i| llvfll Iw^+iu] < cilluW - Qiu\\ + |A - A^^^l ||uW|| 
Furthermore, there exists a c> such that 

I ^ I 
I wtZ 111 

en{i) < c 



1/32/33 •••/?£! 



The proof is deferred to the end of the section. The point here is that 
u^^^ will approach Qeu at a rate governed (for the most part) by how quickly 
|'w|_|_-^u| 0. If e-uii) — > then u^^^ will approach Qeu more rapidly than it 
will converge to u itself. 

Analogous results may be obtained for the Arnoldi method. Suppose A 
has a Schur decomposition A = VHV* for some n x n unitary matrix V and 
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some upper Hessenberg matrix, H. For any index 1 < £ < n, let denote 
the ith principal submatrix of H: 



ai 712 713 

P2 "2 723 



7^-1,. 



(I) 

and define H]^2 and H^+i via a partitioning of H as 



H 



H 



12 



Partition V = [ V^+i ] as before. The Arnoldi method proceeds 
using a recurrence similar to (7.1): 

but here is forced to have orthonormal columns. 

Note that Qi = ILi = V^V| is an orthogonal projection onto the subspace 
Si = span^^;^ ^{vfc}. So = Q^AQi = V^H^V^, and the eigenvalues for 
H^, the so-called "Arnoldi eigenvalues", are Galerkin approximations to the 
eigenvalues of A. In particular, if H^z^^) = A^^^z^^^ and u^^) = V^z^^) then 
{A^^^ u^^)} will be a Galerkin eigenvalue/eigenvector pair ostensibly approxi- 
mating some eigenvalue/eigenvector pair {A, u} of A. 

The expression for e>i(^) becomes 



|Hg(v^,,u) 



sup ■ 



(V. 



|V,%iu|| 



Theorem 7.2 Let X be a simple eigenvalue of A with an associated eigenvec- 
tor u. Suppose the Arnoldi eigenvalue A^^^ converges to X as £ increases. There 
exist constants cq, ci > so that for each i > 1 the eigenvalue/ eigenvector pair 
{A*^^), u^^)} (with u(^) appropriately normalized) satisfies 



co||u*^^^ - Q£U.\\ < 



■■12 



< Cl llu 



Q^ull + |A- A^l 



Furthermore there exists a C2 > so that 



Co- 



nsul I 



< 



|H 



12 ' 



- n.ui 



„(/-n,)u|| - ||v,%,u|| IIU-n,)u| 

where u* is a left unit eigenvector associated with X. 



+ C2||(/-n^)u*| 
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Proof: An easy calculation provides QeA{I — Qe)u = V^V^AV^+iV^ 
Both assertions then follow from (|4.4D and (|4.5| ) of Theorem iA. ■ 



,u. 



Two preliminary results will first be established that may be of some inde- 
pendent interest. The following two lemmas generalize Lemma 4.1 and Lemma 
6.1 of 0. 

Lemma 7.3 Let S be an s- dimensional subspace ofC^ and suppose S G C"^* 
is a given matrix with orthonormal columns so that S = Ran{S). Let T be 
a t-dimensional subspace of C" with t > s. There exists T G ([^nxs ^^^^^ 
orthonormal columns so that Ran{T) C T and 

||S-T|| < V26n{S,T) 

Proof: If SnT is a nontrivial subspace of with dimension k then there is a 
matrix Sq € C"^'^ with orthonormal columns and a unitary matrix Zq € C^'' 
so that SnT = Ran{So) and SZq = [Sq Si] for some choice of Si G C"^ 
having orthonormal columns. (In particular, if t + s > n then k > t + s — n.) 
If 5 n T = {0} then define directly, k = 0, Zq = I, and Si = S. 

Let Ti G C"^^*"'^^ be a matrix with orthonormal columns so that i?an(Ti) = 
r e (5 n T) and let ts G C"^("-*) be chosen so that [Sq ti ts] is unitary. 



The orthogonal projection onto S is given by 11^ 
orthogonal projections onto T and are given by 



[So Si 



SI 



The 



nr = [Soti] ^ ^0 J and 

respectively. Then, 

6n{S,T) =||(I-nr)n5|| 



I-Ut = T2T 



2 J-2) 



ts t^[SoSi 



SI 



msi\ 



Observe that 



TJSi 
T*Si 



G C*^" '^)^('^ has orthonormal columns. By si- 



multaneously diagonalizing the summands in the expression (T|Si)*(T|Si) + 
(T^Si)*(T^Si) = I, one finds a unitary matrix Zi G C^""'')^ and diagonal 
matrices P = diag{'^i, "js-k) £ (£ii-t-k)x(s-k) ^ = diag{ai, ag-k) £ 
t^in-t)xis-k) ^-^j^ 71 > 72 > • • • > 7,_fc, ai < (72 < • • • < as-k, and ^2+72 = 1 
so that 

z];(t];Si)*(t];Si)Zi = p*p 

Zt(t^Si)*(t^Si)Zi = 
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Thus there are unitary matrices Ui G £it-k)xit-k) jj^ ^ (j-(n t)x(n t) 
such that tjSiZi = UiF and t^SiZi = U2S. Thus, 6^(5, T) = ||t^Si|| = 
||S|| = as-k- 

Now let Uii denote the {t — k)x(s — k) matrix consisting of the first s — k 
columns of Ui and define 



So TiUiiZ^ 



Then 



= ||(S-T)Zodm5(I,Zi)||2 

= ||[SoSiZi] - [So tiUii]f = IISiZi - TiUnf 

= max x*(SiZi)*(SiZi)x + x*(tiUii)*(tiUii)x 

||x||=l 

-23f?ex*U];i(t;^SiZi)x 

s—k 

--2 max (1 - 3f?e Vt* \xi\'^) = 2(1 - 7.-A:) 

l|x|I=l ^ 



Since 1 + ^s~k > Ij the conclusion follows from 



|S - T|| < V2(l-7,_fc)(l + 7,_fc) = ^/2(j, 



s—k 



Lemma 7.4 Let lA he the invariant subspace of A associated with the eigen- 
value X. Suppose I is an index such that £ > m = dim U. Then 

1/32/33 •••/?m I < \\W\\{1 + V2)6n{U,ICe{A,vi)) 

Proof: Let Hi denote the orthogonal projection onto /C£(A, vi). Fix for the 
time being a vector u G U with ||u|| = 1 and let p be that polynomial of 
degree £ — 1 or less such that p{A)vi = II^u. Pick e > and (if necessary) 
vary the coefficients of p slightly to obtain a polynomial p so that p{\) 7^ 
and 11(1 — nf)u|| + e > ||u — p(A)vi||. Now, j5(A) maps U bijectively to itself 
so there exists u (zU such that j5(A)u = u. Then 

||(I-n,)u||+e> ||u-p(A)vi|| = ||p(A)(u-vi)||. 

Denote the QR decomposition of W^+i as W^+i = SR so that S has or- 
thonormal columns and i?an(W^_(_i) = Ran{S). Observe that i?an(W^+i) = 
(RanCVi))-^ = /C^(A, vi)^. Lemma |7^ guarantees the existance of a matrix 
T G i[^nx{n-e) ^[^Y^ orthonormal columns and with Ran{T) C Z//"*" such that 

lis - T|| < V2 5niJCe{A,vi)^,U^) = V2 6niK,JCeiA,vi) 
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Note that 



W,Vip(A)vi =W,Vi(Vp(A)W*)Vei 
= [OI„„^]p(J)ei 
=(/32/33---/3m)ei gC("-^^ 

On the other hand, 

W|+iP(A)vi = R*(S - T)*p(A)vi + R*T*p(A)(u - vi) - R*T*p(A)u 

But since T*p(A)u = 0, we find 

1/32/33 •••/3ml =||W,Vp(A)vi|| 

<||R||(||S-T||||p(A)vi|| + ||p(A)(u-vi)|| 
<||W,+i||(^/2 5„(Z^,/C,(A, vi)) + 11(1 - n,)u|| + e) 

The conclusion fohows by letting e ^ 0, taking the supremum over all unit 
vectors u gU, and noticing that ||W£+i|| < ||W||. ■ 



Proof of Theorem ^H]: An easy calculation provides QiA{I — Qi)vL 



7£+iv^w|^-^u. The first assertion then follows from (4.4) of Theorem 
For the second assertion, observe first that since A is simple ||(I — n^)u 
5T-i{U,K.t{A,^i). Then using Lemma ^ we find, 

||Q,A(/-Q,)u|| 

^w(«j =■ 



ll(i-n^)u|| 

r- \^^+^ I llv/ll |w^ , 1 u| 

1/52/33 • • • pi+i\ 

I * I 
lw« , 1 ul 

< c- 



1/32/33 •••/3£|' 
since |7f+i| = |/?f+i|. ■ 

Appendix: Lower Bounds to sep 

Let Wi and W2 be complex Hilbert spaces and denote with £(yV2,yVi) the 
associated Banach space of bounded linear transformations from W2 to Wi. 
Suppose there are given two linear operators, Li : Wi Wi, L2 : VV2 — > VV2 
such that Li is closed, densely defined (but not necessarily bounded) on Wi 
and L2 is bounded (and everywhere defined) in W2. Define an operator T : 
C{W2, Dom{Li)) £(W2, Wi) as T{S) = LiS - SL2 and let 

sep{L,,L2)'=' inf WnS^Wm^^. ^ 

SGC{W2,Dom{Li)) ||i||w2^Wi 
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so that, in particular, if S solves T{S) = M and < 77 < sep{Li, L2) then 



||'S'||w2-^Wi < — 77 — rTll-^ll>V2-^>Vi < -||^||w2-*Wi- 
sep{Li,L2) T] 

The following results are mild generalizations of [^] Theorem 3.1, p. 264 and 
Theorem 5, p. 427. For all e > 0, we define the pseudospectral sets 

A,(L)=|zeC ||(z-L)-1|| > i|. 

As e ^ 0, Ae(-L) shrinks to a{L). 

Lemma A.l Suppose thata{Li) anda{L2) are disjoint. Then for every M £ 
£(yV2,yVi), the operator equation 

LiS-SL2 = M (A.2) 

has a unique solution S that is a bounded linear transformation from W2 to 
Dom{Li) C Wi- That is, T{S) is bijective. 
(a) S has the representation 

S=—[ {Li- z)-^M{z- L2)-^ dz. (A.3) 



2m 



r2 



where T2 consists of a finite number of closed rectifiable Jordan curves enclos- 
ing all points of a{L2) and no points of a{Li). 

(b) Let ei, £2 > 6e chosen so that Ae^(Li)nr2 = and Ae2(L2)nr2 = 0, 
respectively. That is, Ae^(Li) lies entirely outside T2 and A^^{L2) lies entirely 
inside r2. Then 

where length{T2) is the arc length ofT2. 

(c) If the numerical ranges of Li and L2, denoted respectively tv{Li) and 
m{L2), are disjoint sets in C and A = dist{)xi{Li), tv{L2)) > 0, then 

sep{Li,L2)>A (A.5) 

Proof: A trivial extension of Corollary 3.3 of {It] guarantees that there 
are a finite number of closed rectifiable Jordan curves enclosing all points 
of a{L2) and no points of a{Li). We orient each of the curves positively 
(i.e., counterclockwise) and refer to them collectively as r2. Observe that 
r2 C p{Li) n p{L2), so the right hand side of [A.3| is well-defined and maps W2 
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into Dom{Li) C Wi. Since /p (z-La) ^ dz =1,^^ L{z-Li) ^ dz = 



and L{L — z) ^ = I + z[L — z) ^, direct substitution of |A.3| into A. 2 yields 
LiS -SL2 = — [ M(z - Ls)"^ - (Li - zV'^M dz 



M. 



Suppose is the difference between any two solutions of A. 2. Then LiN 
NL2 and 

{z- Li)N = N{z- L2) 
N{z-L2)-'^ = {z-Li)-^N 



N 



A = 0. 



Thus |A.3| gives the one, unique solution to |A.2 . 
To show (b) note that, 

I|5'||w2-^wi < 77- / 11(^1 - 2:)"^ll>Vil|M||w2^>Vill(^; - ^2)""^! 
27r 



W2 \dz\ 



length(r2) 1 
<^^max||(Li-.) II 

^ length(r2) 1 1 

< —\\M ||w2^Wi 



I Ml 



max llfz - Lo) -^1 



Z&T2 



2-K ei 62 

For (c), note that existence and uniqueness of S follows from (a) and 
the observation that disjoint numerical ranges of Li and L2 imply disjoint 
spectra for Li and L2, however, we will use a different representation for S 
to obtain estimates. Define zi, Z2 £ C so that zi £ closure{n}{Li)} , Z2 G 
closure{to{L2)} , and 



A = \zi - Z2I 



: inf l^^i — 22I 

zi6w(Li) 
Z2&Kl{L2) 



{x, Lix)w, _ {y, L2y)w2 

{y, y)m 



: mf 

J/GW2 



(A.6) 
(A.7) 



Define 9 G [0, 27r) as 6 = arg{zi — Z2). Let Li = e {Li — zi) and L2 - 
e-'\L2 - h). Then tt)(Li) = e-'\w{Li) - h) and tt)(L2) = e-'^(tt3(L2) - zi] 
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The goal of this translation and rotation is to insure 

5Re (x, Li2;)yvi > 

for all X G Dom{Li), and 

{y, l/22/)w2 < -^\\y\\m 

for all y € W2). 

Under these circumstances, the Lumer-Phillips theorem (cf. [15|, §1.4) 
guarantees that —Li and L2 each generate strongly continuous, one parameter 
semigroups, e~*^^ and e*^^, respectively. Furthermore, ||e~*^^ ||v\;;^ < 1 and 

We^^^Wm < e"*^ for all t > 0. 

Notice that S satisfies (|A.2D if and only if it is also a solution to L1S—SL2 = 
e~*^M and this leads to the following representation for S: 

roo 

S = e-'^ e~*^'Me*^^ dt. (A.8) 
Indeed, note that with this expression for S, we have for any v G W2 
II {LiS - SL2 - M) v\\w, = II {LiS - SL2 - e-'^M^ v\\w^ 

Lie-*^We*^2^ - e-^^'Me^^^L2V dt - Mv\\y^^ 







00 1 

ie-^^^Me^^^v] dt-Mv\ 

dt 



lim (e'^^^Mv - Mv] + lime"*^W (e 



Wi 

tL2V~v„ 



lim (e-^^^Me^^^i 



t— >oo \ / 

< lini \\{e-^^^Mv - Mv\\yv^ + ||M||w2^>Vi Jim \\e^^^v - v||wi 
+ lim ||M||w2^>Vie~*^||v||>V2 =0. 



Immediately then, one obtains 



00 



S\\w2^m< I ||e-'^MIwil|M||w2-Wil|e*^2||w2 (A.9) 


< / e-*^ dt ||M||w2-.Wi = ( X ) l|M||w2-Wi 

(A.IO) 
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